Mechanism for comparison of disparate data in data structures

ABSTRACT

A device receives a first result that includes first data, and a second result that includes second data, and determines whether a comparator supports the first data and the second data. When the comparator supports the first data and the second data, the device utilizes the comparator to select comparison logic for the first data of the first result and for the second data of the second result, compares the first result and the second result, using the selected comparison logic, to determine whether the first result is equivalent to the second result, and outputs or stores the determination of whether the first result is equivalent to the second result.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate one or more implementations and,together with the description, explain these implementations. In thedrawings:

FIG. 1 is a diagram of an overview of an example implementationdescribed herein;

FIG. 2 is a diagram of an example environment in which systems and/ormethods described herein may be implemented;

FIG. 3 is a diagram of example components of one or more of the devicesof the environment depicted in FIG. 2;

FIG. 4 is a diagram of example functional components of a technicalcomputing environment (TCE) that may be used by one or more of thedevices of the environment depicted in FIG. 2;

FIG. 5 is a diagram of example operations capable of being performed bythe TCE;

FIG. 6 is a diagram of example functional components of a testingcomponent of the TCE;

FIG. 7 is a diagram of example operations capable of being performed bythe testing component;

FIG. 8 is a diagram of example operations capable of being performed bya comparator component of the testing component; and

FIGS. 9 and 10 are flow charts of an example process for comparingdisparate data in arbitrarily complex data structures.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements.

A technical computing environment (TCE) may provide a computingenvironment that allows users to perform tasks related to disciplines,such as, but not limited to, mathematics, science, engineering,medicine, business, etc., more efficiently than if the tasks wereperformed in another type of computing environment, such as anenvironment that requires the user to develop code in a conventionalprogramming language, such as C++, C, Fortran, Pascal, etc. In oneexample, a TCE may include a dynamically-typed programming language(e.g., the M language, a MATLAB® language, a MATLAB-compatible language,a MATLAB-like language, etc.) that can be used to express problemsand/or solutions in mathematical notations.

Code generated by the TCE may be tested to determine whether the codewill function properly (e.g., when executed). In a simple example, thecode may be output as a simulation and an executable. The simulationcode may be executed to generate simulation results, and the executablecode may be executed to generate executable results. The simulationresults and the executable results may be tested to determine whetherthe simulation results are equivalent to the executable results.However, comparing such results may be difficult when the resultsinclude disparate data in complex data structures.

OVERVIEW

Systems and/or methods described herein may compare disparate data inarbitrarily complex data structures in order to determine whether thedata structures are equivalent (e.g., pass or fail a qualificationtest). In one example, the systems and/or methods may provide acomparator application programming interface (API) that enables a testerof program code to compare complex data structures (e.g., cell arrays,structure arrays, etc.). The data structures may include many differenttypes of data and each data type may require different methods ofcomparison. The single comparator API may recursively compare allelements of a data structure so that each element may utilize comparisonlogic that is appropriate for that element.

FIG. 1 is a diagram of an overview of an example implementationdescribed herein. As shown in FIG. 1, a computing environment, such as atechnical computing environment (TCE), may include a testing component.The testing component may receive results generated by TCE code, and maytest the results to determine whether the code functions properly.

As further shown in FIG. 1, the testing component may receive resultsgenerated by TCE code. The TCE code may include text-based code that mayrequire further processing to execute, binary code that may be executed,text files that may be executed in conjunction with other executables,etc. In one example, the TCE code may be executed to generate a firstresult that includes first data. The TCE code may again be executed togenerate a second result that includes second data. In one example, thefirst result may be generated by a simulation version of the TCE code,and the second result may be generated by an executable version of theTCE code. The first data and the second data may be the same type ordifferent types, and may include floating point values, cell arrays,structure arrays, simulation output data, etc. The testing component mayreceive a test case to be applied to the first result and the secondresult. For example, the test case may include a test (e.g., IsEqualTo)to determine whether the first result is equivalent to the secondresult.

The testing component may determine whether a comparator, of the testingcomponent, supports the first data and the second data. If thecomparator does not support the first data and/or the second data, thetesting component may provide (e.g., display) an indication of anunsupported data in the first result and/or the second result. If thecomparator supports the first data and the second data, the testingcomponent may utilize the comparator to select comparison logic that isappropriate for the first data of the first result and the second dataof the second result.

In one example implementation, the comparator may include a set (e.g., alist) of comparison logic that may be used to support different datastructures. For example, the list may include comparison logic for cellarrays, structure arrays, numeric data, logical data, string data,object data, etc. The testing component may determine first comparisonlogic of the list that matches the first data, and may determine secondcomparison logic of the list that matches the second data. Thecomparator may output the first comparison logic as the selectedcomparison logic that is appropriate for the first data of the firstresult. The comparator may output the second comparison logic as theselected comparison logic that is appropriate for the second data of thesecond result.

The testing component may compare the first result and the secondresult, using the selected comparison logic and based on the test case,to determine whether the first result is equal to the second result. Thetesting component may output a determination of whether the first resultis equal to the second result. If the first result equals the secondresult, the testing component may output (e.g., display) and/or storediagnostic information indicating the equivalence of the first resultand the second result. For example, as shown in FIG. 1, the diagnosticinformation may state: “The first result IsEqualTo the second result.”If the first result does not equal the second result, the testingcomponent may output (e.g., display) and/or store diagnostic informationindicating the nonequivalence of the first result and the second result.

The terms “code” and “program code,” as used herein, are to be usedinterchangeably and are to be broadly interpreted to include text-basedcode (e.g., C++ code, Hardware Description Language (HDL) code,very-high-speed integrated circuits (VHSIC) HDL(VHDL) code, Verilog,Java, and/or other types of hardware or software based code that may becompiled and/or synthesized); binary code that may be executed (e.g.,executable files that may directly be executed by an operating system,bitstream files that can be used to configure a field programmable gatearray (FPGA), Java byte code, object files combined together with linkerdirectives, source code, makefiles, etc.); text files that may beexecuted in conjunction with other executables (e.g., Python text files,a collection of dynamic-link library (DLL) files with text-basedcombining, configuration information that connects pre-compiled modules,an extensible markup language (XML) file describing module linkage,etc.); etc. In one example, code may include different combinations ofthe above-identified classes (e.g., text-based code, binary code, textfiles, etc.). Alternatively, or additionally, code may include codegenerated using a dynamically-typed programming language (e.g., the Mlanguage, a MATLAB® language, a MATLAB-compatible language, aMATLAB-like language, etc.) that can be used to express problems and/orsolutions in mathematical notations. Alternatively, or additionally,code may be of any type, such as function, script, object, etc., and aportion of code may include one or more characters, lines, etc. of thecode.

Example Environment Arrangement

FIG. 2 is a diagram of an example environment 200 in which systemsand/or methods described herein may be implemented. As illustrated,environment 200 may include a client device 210 interconnected with aserver device 220 via a network 230. Components of environment 200 mayinterconnect via wired and/or wireless connections. A single clientdevice 210, server device 220, and network 230 have been illustrated inFIG. 2 for simplicity. In practice, environment 200 may include moreclient devices 210, server devices 220, and/or networks 230. In oneexample implementation, client device 210 and server device 220 may beprovided in a single device or may be provided in separate devices.

Client device 210 may include one or more devices that are capable ofcommunicating with server device 220 via network 230. For example,client device 210 may include a laptop computer, a personal computer, atablet computer, a desktop computer, a workstation computer, a smartphone, a personal digital assistant (PDA), and/or other computation andcommunication devices.

Server device 220 may include one or more server devices, or other typesof computation and communication devices, that gather, process, and/orprovide information in a manner described herein. Server device 220 mayinclude a device that is capable of communicating with client device 210(e.g., via network 230). In one example, server device 220 may includeone or more laptop computers, personal computers, workstation computers,servers, central processing units (CPUs), graphical processing units(GPUs), application-specific integrated circuits (ASICs),field-programmable gate arrays (FPGAs), etc. and/or software (e.g., asimulator) executing on the aforementioned devices. In one example,server device 220 may include TCE 240 and may perform some or all of thefunctionality described herein for client device 210. Alternatively,server device 220 may be omitted and client device 210 may perform allof the functionality described herein for client device 210.

Network 230 may include a network, such as a local area network (LAN), awide area network (WAN), a metropolitan area network (MAN), a telephonenetwork, such as the Public Switched Telephone Network (PSTN), anintranet, the Internet, or a combination of networks.

TCE 240 may be provided within a computer-readable medium of clientdevice 210. Alternatively, or additionally, TCE 240 may be provided inanother device (e.g., server device 220) that is accessible by clientdevice 210. TCE 240 may include hardware or a combination of hardwareand software that provides a computing environment that allows users toperform tasks related to disciplines, such as, but not limited to,mathematics, science, engineering, medicine, business, etc., moreefficiently than if the tasks were performed in another type ofcomputing environment, such as an environment that required the user todevelop code in a conventional programming language, such as C++, C,Fortran, Pascal, etc. In one implementation, TCE 240 may include adynamically-typed programming language (e.g., the M language, a MATLAB®language, a MATLAB-compatible language, a MATLAB-like language, etc.)that can be used to express problems and/or solutions in mathematicalnotations.

For example, TCE 240 may use an array as a basic element, where thearray may not require dimensioning. These arrays may be used to supportarray-based programming where an operation may apply to an entire set ofvalues included in the arrays. Array-based programming may allowarray-based operations to be treated as high-level programming that mayallow, for example, operations to be performed on entire aggregations ofdata without having to resort to explicit loops of individual non-arrayoperations. In addition, TCE 240 may be adapted to perform matrix and/orvector formulations that can be used for data analysis, datavisualization, application development, simulation, modeling, algorithmdevelopment, etc. These matrix and/or vector formulations may be used inmany areas, such as statistics, image processing, signal processing,control design, life sciences modeling, discrete event analysis and/ordesign, state based analysis and/or design, etc.

TCE 240 may further provide mathematical functions and/or graphicaltools (e.g., for creating plots, surfaces, images, volumetricrepresentations, etc.). In one implementation, TCE 240 may provide thesefunctions and/or tools using toolboxes (e.g., toolboxes for signalprocessing, image processing, data plotting, parallel processing, etc.).Alternatively, or additionally, TCE 240 may provide these functions asblock sets or in another way, such as via a library, etc.

TCE 240 may be implemented as a text-based environment (e.g., MATLABsoftware; Octave; Python; Comsol Script; MATRIXx from NationalInstruments; Mathematica from Wolfram Research, Inc.; Mathcad fromMathsoft Engineering & Education Inc.; Maple from Maplesoft; Extend fromImagine That Inc.; Scilab from The French Institution for Research inComputer Science and Control (INRIA); Virtuoso from Cadence; Modelica orDymola from Dynasim; etc.); a graphically-based environment (e.g.,Simulink® software, Stateflow® software, SimEvents® software, Simscape™software, etc., by The MathWorks, Inc.; VisSim by Visual Solutions;LabView® by National Instruments; Dymola by Dynasim; SoftWIRE byMeasurement Computing; WiT by DALSA Coreco; VEE Pro or SystemVue byAgilent; Vision Program Manager from PPT Vision; Khoros from KhoralResearch; Gedae by Gedae, Inc.; Scicos from (INRIA); Virtuoso fromCadence; Rational Rose from IBM; Rhopsody or Tau from Telelogic; Ptolemyfrom the University of California at Berkeley; aspects of a UnifiedModeling Language (UML) or SysML environment; etc.); or another type ofenvironment, such as a hybrid environment that includes one or more ofthe above-referenced text-based environments and one or more of theabove-referenced graphically-based environments.

TCE 240 may include a programming language (e.g., the MATLAB language)that may be used to express problems and/or solutions in mathematicalnotations. The programming language may be dynamically typed and/orarray-based. In a dynamically typed array-based computing language, datamay be contained in arrays and data types of the data may be determined(e.g., assigned) at program execution time.

For example, suppose a program, written in a dynamically typedarray-based computing language, includes the following statements:

-   -   A=‘hello’    -   A=int32([1, 2])    -   A=[1.1, 2.2, 3.3].

Now suppose the program is executed, for example, in a TCE, such as TCE240. During run-time, when the statement “A=‘hello’” is executed thedata type of variable “A” may be a string data type. Later when thestatement “A=int32([1, 2])” is executed the data type of variable “A”may be a 1-by-2 array containing elements whose data type are 32 bitintegers. Later, when the statement “A=[1.1, 2.2, 3.3]” is executed,since the language is dynamically typed, the data type of variable “A”may be changed from the above 1-by-2 array to a 1-by-3 array containingelements whose data types are floating point. As can be seen by thisexample, data in a program written in a dynamically typed array-basedcomputing language may be contained in an array. Moreover, the data typeof the data may be determined during execution of the program. Thus, ina dynamically type array-based computing language, data may berepresented by arrays and data types of data may be determined atrun-time.

TCE 240 may provide mathematical routines and a high-level programminglanguage suitable for non-professional programmers and may providegraphical tools that may be used for creating plots, surfaces, images,volumetric representations, or other representations. TCE 240 mayprovide these routines and/or tools using toolboxes (e.g., toolboxes forsignal processing, image processing, data plotting, parallel processing,etc.). TCE 240 may also provide these routines in other ways, such as,for example, via a library, local or remote database (e.g., a databaseoperating in a computing cloud), remote procedure calls (RPCs), and/oran application programming interface (API). TCE 240 may be configured toimprove runtime performance when performing computing operations. Forexample, TCE 240 may include a just-in-time (JIT) compiler.

Although FIG. 2 shows example components of environment 200, in otherimplementations, environment 200 may include fewer components, differentcomponents, differently arranged components, and/or additionalcomponents than those depicted in FIG. 2. Alternatively, oradditionally, one or more components of environment 200 may perform oneor more other tasks described as being performed by one or more othercomponents of environment 200.

Example Device Architecture

FIG. 3 is an example diagram of a device 300 that may correspond to oneor more of the devices of environment 200. As illustrated, device 300may include a bus 310, a processing unit 320, a main memory 330, aread-only memory (ROM) 340, a storage device 350, an input device 360,an output device 370, and/or a communication interface 380. Bus 310 mayinclude a path that permits communication among the components of device300.

Processing unit 320 may include one or more processors, microprocessors,or other types of processing units that may interpret and executeinstructions. Main memory 330 may include one or more random accessmemories (RAMs) or other types of dynamic storage devices that may storeinformation and/or instructions for execution by processing unit 320.ROM 340 may include one or more ROM devices or other types of staticstorage devices that may store static information and/or instructionsfor use by processing unit 320. Storage device 350 may include amagnetic and/or optical recording medium and its corresponding drive.

Input device 360 may include a mechanism that permits a user to inputinformation to device 300, such as a keyboard, a camera, anaccelerometer, a gyroscope, a mouse, a pen, a microphone, voicerecognition and/or biometric mechanisms, a remote control, a touchscreen, a neural interface, etc. Output device 370 may include amechanism that outputs information to the user, including a display, aprinter, a speaker, etc. Communication interface 380 may include anytransceiver-like mechanism that enables device 300 to communicate withother devices, networks, and/or systems. For example, communicationinterface 380 may include mechanisms for communicating with anotherdevice or system via a network.

As described herein, device 300 may perform certain operations inresponse to processing unit 320 executing software instructionscontained in a computer-readable medium, such as main memory 330. Acomputer-readable medium may be defined as a non-transitory memorydevice. A memory device may include space within a single physicalmemory device or spread across multiple physical memory devices. Thesoftware instructions may be read into main memory 330 from anothercomputer-readable medium, such as storage device 350, or from anotherdevice via communication interface 380. The software instructionscontained in main memory 330 may cause processing unit 320 to performprocesses described herein. Alternatively, hardwired circuitry may beused in place of or in combination with software instructions toimplement processes described herein. Thus, implementations describedherein are not limited to any specific combination of hardware circuitryand software.

Although FIG. 3 shows example components of device 300, in otherimplementations, device 300 may include fewer components, differentcomponents, differently arranged components, and/or additionalcomponents than depicted in FIG. 3. Alternatively, or additionally, oneor more components of device 300 may perform one or more other tasksdescribed as being performed by one or more other components of device300.

Example Technical Computing Environment

FIG. 4 is a diagram of example functional components of TCE 240. In oneimplementation, the functions described in connection with FIG. 4 may beperformed by one or more components of device 300 (FIG. 3) and/or by oneor more devices 300. As shown in FIG. 4, TCE 240 may include a blockdiagram editor 410, graphical entities 420, blocks 430, and/or anexecution engine 440.

Block diagram editor 410 may include hardware or a combination ofhardware and software that may be used to graphically specify models ofdynamic systems. In one implementation, block diagram editor 410 maypermit a user to perform actions, such as construct, edit, display,annotate, save, and/or print a graphical model (e.g., a block diagramthat visually and/or pictorially represents a dynamic system). Inanother implementation, block diagram editor 410 may permit a user tocreate and/or store data relating to graphical entities 420.

A textual interface may be provided to permit interaction with blockdiagram editor 410. A user may write scripts that perform automaticediting operations on a model using the textual interface. For example,the textual interface may provide a set of windows that may act as acanvas for the model, and may permit user interaction with the model. Amodel may include one or more windows depending on whether the model ispartitioned into multiple hierarchical levels.

Graphical entities 420 may include hardware or a combination of hardwareand software that may provide entities (e.g., signal lines, buses, etc.)that represent how data may be communicated between functional and/ornon-functional units and blocks 430 of a model. Blocks 430 may includefundamental mathematical elements of a block diagram model.

Execution engine 440 may include hardware or a combination of hardwareand software that may process a graphical model to produce simulationresults, may convert the graphical model into executable code, and/ormay perform other analyses and/or related tasks. In one implementation,for a block diagram graphical model, execution engine 440 may translatethe block diagram into executable entities (e.g., units of execution)following the layout of the block diagram. The executable entities maybe compiled and/or executed on a device (e.g., client device 210) toimplement the functionality specified by the model.

Graphical models may include entities with relationships between theentities, and the relationships and/or the entities may have attributesassociated with them. The entities my include model elements such asblocks 430 and ports. The relationships may include model elements suchas lines (e.g., connector lines) and references. The attributes mayinclude model elements such as value information and meta informationfor the model element associated with the attributes. Graphical modelsmay be associated with configuration information. The configurationinformation may include information for the graphical model such asmodel execution information (e.g., numerical integration schemes,fundamental execution period, etc.), model diagnostic information (e.g.,whether an algebraic loop should be considered an error or result in awarning), model optimization information (e.g., whether model elementsshould share memory during execution), model processing information(e.g., whether common functionality should be shared in code that isgenerated for a model), etc.

Additionally, or alternatively, a graphical model may have executablesemantics and/or may be executable. An executable graphical model may bea time based block diagram. A time based block diagram may consist, forexample, of blocks (e.g., blocks 430) connected by lines (e.g.,connector lines). The blocks may consist of elemental dynamic systemssuch as a differential equation system (e.g., to specify continuous-timebehavior), a difference equation system (e.g., to specify discrete-timebehavior), an algebraic equation system (e.g., to specify constraints),a state transition system (e.g., to specify finite state machinebehavior), an event based system (e.g., to specify discrete eventbehavior), etc. The lines may represent signals (e.g., to specifyinput/output relations between blocks or to specify executiondependencies between blocks), variables (e.g., to specify informationshared between blocks), physical connections (e.g., to specifyelectrical wires, pipes with volume flow, rigid mechanical connections,etc.), etc. The attributes may consist of meta information such assample times, dimensions, complexity (whether there is an imaginarycomponent to a value), data type, etc. associated with the modelelements.

In a time based block diagram, ports may be associated with blocks(e.g., blocks 430). A relationship between two ports may be created byconnecting a line (e.g., a connector line) between the two ports. Linesmay also, or alternatively, be connected to other lines, for example bycreating branch points. For instance, three or more ports can beconnected by connecting a line to each of the ports, and by connectingeach of the lines to a common branch point for all of the lines. Acommon branch point for the lines that represent physical connectionsmay be a dynamic system (e.g., by summing all variables of a certaintype to 0 or by equating all variables of a certain type). A port may bean input port, an output port, an enable port, a trigger port, afunction-call port, a publish port, a subscribe port, an exception port,an error port, a physics port, an entity flow port, a data flow port, acontrol flow port, etc.

Relationships between blocks (e.g., blocks 430) may be causal and/ornon-causal. For example, a model may include a block that represents acontinuous-time integration block that may be causally related to a datalogging block by using a line (e.g., a connector line) to connect anoutput port of the continuous-time integration block to an input port ofthe data logging block. Further, during execution of the model, thevalue stored by the continuous-time integrator may change as the currenttime of the execution progresses. The value of the state of thecontinuous-time integrator may be available on the output port and theconnection with the input port of the data logging block may make thisvalue available to the data logging block.

A sample time may be associated with the elements of a graphical model.For example, a graphical model may include a block (e.g., block 430)with a continuous sample time such as a continuous-time integrationblock that may integrate an input value as time of execution progresses.This integration may be specified by a differential equation. Duringexecution the continuous-time behavior may be approximated by anumerical integration scheme that is part of a numerical solver. Thenumerical solver may take discrete steps to advance the execution time,and these discrete steps may be constant during an execution (e.g.,fixed step integration) or may be variable during an execution (e.g.,variable-step integration).

Alternatively, or additionally, a graphical model may include a block(e.g., block 430) with a discrete sample time such as a unit delay blockthat may output values of a corresponding input after a specific delay.This delay may be a time interval and this interval may determine asample time of the block. During execution, the unit delay block may beevaluated each time the execution time has reached a point in time wherean output of the unit delay block may change. These points in time maybe statically determined based on a scheduling analysis of the graphicalmodel before starting execution.

Alternatively, or additionally, a graphical model may include a block(e.g., block 430) with an asynchronous sample time, such as afunction-call generator block that may schedule a connected block to beevaluated at a non-periodic time. During execution, a function-callgenerator block may evaluate an input and when the input attains aspecific value when the execution time has reached a point in time, thefunction-call generator block may schedule a connected block to beevaluated at this point in time and before advancing execution time.

Further, the values of attributes of a graphical model may be inferredfrom other elements of the graphical model or attributes of thegraphical model. For example, the graphical model may include a block(e.g., block 430), such as a unit delay block, that may have anattribute that specifies a sample time of the block. When a graphicalmodel has an execution attribute that specifies a fundamental executionperiod, the sample time of the unit delay block may be inferred fromthis fundamental execution period.

As another example, the graphical model may include two unit delayblocks (e.g., blocks 430) where the output of the first of the two unitdelay blocks is connected to the input of the second of the two unitdelay block. The sample time of the first unit delay block may beinferred from the sample time of the second unit delay block. Thisinference may be performed by propagation of model element attributessuch that after evaluating the sample time attribute of the second unitdelay block, a graph search proceeds by evaluating the sample timeattribute of the first unit delay block since it is directly connectedto the second unit delay block.

The values of attributes of a graphical model may be set tocharacteristics settings, such as one or more inherited settings, one ormore default settings, etc. For example, the data type of a variablethat is associated with a block (e.g., block 430) may be set to adefault such as a double. Because of the default setting, an alternatedata type (e.g., a single, an integer, a fixed point, etc.) may beinferred based on attributes of elements that the graphical modelcomprises (e.g., the data type of a variable associated with a connectedblock) and/or attributes of the graphical model. As another example, thesample time of a block may be set to be inherited. In case of aninherited sample time, a specific sample time may be inferred based onattributes of elements that the graphical model comprises and/orattributes of the graphical model (e.g., a fundamental executionperiod).

Although FIG. 4 shows example functional components of TCE 240, in otherimplementations, TCE 240 may include fewer functional components,different functional components, differently arranged functionalcomponents, and/or additional functional components than depicted inFIG. 4. Alternatively, or additionally, one or more functionalcomponents of TCE 240 may perform one or more other tasks described asbeing performed by one or more other functional components of TCE 240.

Example Technical Computing Environment Operations

FIG. 5 is a diagram of example operations 500 capable of being performedby TCE 240. TCE 240 may include the features described above inconnection with, for example, one or more of FIGS. 1-4. As illustratedin FIG. 5, TCE 240 may include a testing component 510. The functionsdescribed in connection with testing component 510 may be performed byone or more components of device 300 (FIG. 3) and/or by one or moredevices 300.

As further shown in FIG. 5, testing component 510 may receive resultsgenerated by TCE code 520. TCE code 520 may include text-based code thatmay require further processing to execute, binary code that may beexecuted, text files that may be executed in conjunction with otherexecutables, etc. In one example, TCE code 520 may be executed togenerate a first result 530 that includes first data. TCE code 520 mayagain be executed to generate a second result 540 that includes seconddata. In one example, first result 530 may be generated by a simulationversion of TCE code 520, and second result 540 may be generated by anexecutable version of TCE code 520. The first data and the second datamay be the same type or different types, and may include floating pointvalues, fixed point values, cell arrays, structure arrays, simulationoutput data, numeric data, logical data, string data, object data, etc.Testing component 510 may receive a test case 550 to be applied to firstresult 530 and second result 540. For example, test case 550 may includea test (e.g., IsEqualTo) to determine whether first result 530 isequivalent to second result 540.

Testing component 510 may determine whether a comparator, of testingcomponent 510, supports the first data and the second data. If thecomparator does not support the first data and/or the second data,testing component 510 may provide (e.g., display) an indication ofunsupported data in first result 530 and/or second result 540. If thecomparator supports the first data and the second data, testingcomponent 510 may utilize the comparator to select comparison logic thatis appropriate for the first data of first result 530 and the seconddata of second result 540.

In one example implementation, the comparator may include a set (e.g., alist) of comparison logic that may be used to support different data.For example, the list may include comparison logic for cell arrays,structure arrays, numeric data, logical data, string data, object data,etc. Testing component 510 may determine first comparison logic of thelist that matches the first data, and may determine second comparisonlogic of the list that matches the second data. The comparator mayoutput the first comparison logic as the selected comparison logic thatis appropriate for the first data of first result 530. The comparatormay output the second comparison logic as the selected comparison logicthat is appropriate for the second data of second result 540.

In one example, the data structures handled by the comparator may needto be compared in a numerically stable manner. Such data structures mayinclude numerical values, such as floating point values, and valuesinside more complex data structures, such as cell arrays, structurearrays, output data, recursive data, etc. The comparator may provide thenumerical comparison of such complex and arbitrary data structures.

Testing component 510 may compare first result 530 and second result540, using the selected comparison logic and based on test case 550, todetermine whether first result 530 is equal to second result 540.Testing component 510 may output a determination 560 of whether firstresult 530 is equal to second result 540. If first result 530 equalssecond result 540, testing component 510 may output (e.g., display)and/or store, as determination 560, diagnostic information 570indicating the equivalence of first result 530 and second result 540.For example, as shown in FIG. 5, diagnostic information 570 may state:“The first result IsEqualTo the second result.” If first result 530 doesnot equal second result 540, testing component 510 may output (e.g.,display) and/or store, as determination 560, diagnostic information 570indicating the nonequivalence of first result 530 and second result 540.Diagnostic information 570 may provide information identifying alocation of the nonequivalence in first result 530 and/or second result540. The location information may include a diagnostic code (e.g., arecursive path) that identifies where the nonequivalence occurs in firstresult 530 and/or second result 540.

In order to compare disparate data structures, typical testing systemsdefine a method (e.g., an “equals” method) for each different datastructure generated by code (e.g., TCE code 520). This may requiremodification of the code prior to testing. In contrast, testingcomponent 510 may permit a different definition of equality to be usedduring testing than used during execution of TCE code 520. Such anarrangement may enable testing component 510 to utilize differentcomparison logic during the test of first result 530 and second result540, without the need to modify TCE code 520 prior to testing.Furthermore, testing component 510 may permit first result 530 andsecond result 540 to be compared, while typical testing systems do notpermit such a comparison. In one example implementation, testingcomponent 510 may recursively reuse the comparison logic for manydifferent data structures structured within complex data structures.This may prevent the comparator from having to re-implement a recursivestep.

In one example implementation, the comparator may include APIs ofabstract methods (e.g., satisfiedBy, getDiagnosticFor, supports, etc.).Alternatively, or additionally, the comparator may include a recursiveAPI (e.g., ContainerComparator) and a composite API (e.g.,ComparatorList) that enable testing component 510 to recursively use thecomparison logic on complex data structures.

Although FIG. 5 shows example operations capable of being performed byTCE 240, in other implementations, TCE 240 may perform fewer operations,different operations, and/or additional operations than depicted in FIG.5. Alternatively, or additionally, one or more components of FIG. 5 mayperform one or more other tasks described as being performed by one ormore other components of FIG. 5.

Example Testing Component Operations

FIG. 6 is a diagram of example functional components of testingcomponent 510 (FIG. 5). The functions described in connection withtesting component 510 may be performed by one or more components ofdevice 300 (FIG. 3) and/or by one or more devices 300. As shown in FIG.6, testing component 510 may include a comparator component 600 anddetermination component 610.

Comparator component 600 may provide an API that can be used inconjunction with a test case (e.g., an IsEqualTo constraint) and thatmay allow a tester to determine comparison logic to use for the test(e.g., the equality comparison provided by the IsEqualTo constraint). Asshown in FIG. 6, comparator component 600 may receive first result 530and second result 540, and may determine whether comparator component600 supports the first data of first result 530 and the second data ofsecond result 540.

If comparator component 600 does not support the first data of firstresult 530 and/or the second data of second result 540, comparatorcomponent 600 may provide (e.g., display) an indication 620 ofunsupported data in first result 530 and/or second result 540. Ifcomparator component 600 supports the first data of first result 530 andthe second data of second result 540, comparator component 600 mayselect comparison logic 630 that is appropriate for the first data offirst result 530 and the second data of second result 540. Comparatorcomponent 600 may provide comparison logic 630 to determinationcomponent 610. Comparison logic 630 may include comparison methods thatmay be applied to the first data of first result 530 and the second dataof second result 540.

Determination component 610 may receive first result 530, second result540, test case 550, and comparison logic 630. Determination component610 may compare first result 530 and second result 540, using theselected comparison logic 630 and based on test case 550, to determinewhether first result 530 is equal to second result 540. Determinationcomponent 610 may output determination 560 of whether first result 530is equal to second result 540. If first result 530 equals second result540, determination component 610 may output (e.g., display) and/orstore, as determination 560, diagnostic information indicating theequivalence of first result 530 and second result 540. If first result530 does not equal second result 540, determination component 610 mayoutput (e.g., display) and/or store, as determination 560, diagnosticinformation indicating the nonequivalence of first result 530 and secondresult 540.

Although FIG. 6 shows example functional components of testing component510, in other implementations, testing component 510 may include fewerfunctional components, different functional components, differentlyarranged functional components, and/or additional functional componentsthan depicted in FIG. 6. Alternatively, or additionally, one or morefunctional components of testing component 510 may perform one or moreother tasks described as being performed by one or more other functionalcomponents of testing component 510.

FIG. 7 is a diagram of example operations 700 capable of being performedby testing component 510. Testing component 510 may include the featuresdescribed above in connection with, for example, one or more of FIGS. 1,5, and 6. As illustrated in FIG. 7, testing component 510 may receive afirst result (a) 710 and a second result (b) 720 generated by code, suchas TCE code 520. In one example, first result 710 may include a firstdata, and second result 720 may include a second data. The first dataand the second data may be the same type or different types, and mayinclude floating point values, cell arrays, structure arrays, simulationoutput data, numeric data, logical data, string data, object data, etc.

As further shown in FIG. 7, testing component 510 may receive a testcase 730 to be applied to first result 710 and second result 720. Forexample, test case 730 may include a test (e.g., testCase.assertThat) todetermine whether first result 710 is equivalent to second result 720.Testing component 510 may generate a test 740 based on first result 710,second result 720, and test case 730. For example, testing component 510may generate test 740 with the syntax testCase.assertThat(a,IsEqualTo(b, ‘Using’, CustomComparator)). Test 740 may determine whetherfirst result (a) 710 is equal to second result (b) 720 using a customcomparator (e.g., CustomComparator).

Testing component 510 may determine whether the custom comparatorsupports the first data of first result 710 and the second data ofsecond result 720. If the custom comparator does not support the firstdata and/or the second data, testing component 510 may provide (e.g.,display) an indication 750 of unsupported data in first result 710and/or second result 720. If the custom comparator supports the firstdata and the second data, testing component 510 may utilize the customcomparator to select comparison logic that is appropriate for the firstdata of first result 710 and the second data of second result 720.

Testing component 510 may compare first result 710 and second result720, using the selected comparison logic and based on test 740, todetermine whether first result 710 is equal to second result 720. Iffirst result 710 equals second result 720, testing component 510 mayoutput (e.g., display) and/or store information 760 (e.g., a=b)indicating the equivalence of first result 710 and second result 720. Iffirst result 710 does not equal second result 720, testing component 510may output (e.g., display) and/or store information 770 (e.g., a≠b)indicating the nonequivalence of first result 710 and second result 720.

Although FIG. 7 shows example operations capable of being performed bytesting component 510, in other implementations, testing component 510may perform fewer operations, different operations, and/or additionaloperations than depicted in FIG. 7. Alternatively, or additionally, oneor more components of FIG. 7 may perform one or more other tasksdescribed as being performed by one or more other components of FIG. 7.

FIG. 8 is a diagram of example operations 800 capable of being performedby comparator component 600. Comparator component 600 may include thefeatures described above in connection with, for example, one or more ofFIGS. 1 and 5-7. As shown in FIG. 8, comparator component 600 mayinclude a comparator list 810 with multiple comparison logic 830. In oneexample, comparison logic 830 of comparator list 810 may handle severaldifferent data structures received by testing component 510 for testing.For example, comparator list 810 may include comparison logic 830 forcell arrays (e.g., CellComparator), comparison logic 830 for structurearrays (e.g., StructComparator), comparison logic 830 for numeric data(e.g., NumericComparator), comparison logic 830 for logical data (e.g.,LogicalComparator), comparison logic 830 for string data (e.g.,StringComparator), comparison logic 830 for object data (e.g.,ObjectComparator), etc.

In one example implementation, comparator component 600 may receive aresult that includes particular data, and may determine whethercomparator list 810 includes comparison logic 830 that supports theparticular data. If comparator list 810 includes comparison logic 830that supports the particular data, comparator component 600 may selectthe first comparison logic 830, from comparator list 810, which supportsthe particular data. Alternatively, or additionally, if comparator list810 includes multiple comparison logic 830 that supports the particulardata, comparator component 600 may select one of the multiple comparisonlogic 830 from comparator list 810.

Some data structures, such as cell arrays and structure arrays, maycontain arbitrarily complex elements that also need to be compared. Forsuch data structures, comparator 600 may recursively select comparisonlogic 830, from comparator list 810, for the different complex elements.For example, as shown in FIG. 8, comparator 600 may receive a result 820that includes the syntax data={5, ‘string’, inputParser}. Comparator 600may determine that data is a cell array, and may select comparison logic830 for cell arrays (e.g., CellComparator). Comparator 600 may examineeach element of the data cell array, and may determine that a firstelement (e.g., ‘5’) is a double value. Since comparison logic 830 forcell arrays (e.g., CellComparator) and comparison logic 830 forstructure arrays (e.g., StructComparator) do not support double values,comparator 600 may select, for the first element, comparison logic 830for numeric data (e.g., NumericComparator). Comparator 600 may determinethat a second element (e.g., string) is string data, and may select, forthe second element, comparison logic 830 for string data (e.g.,StringComparator). Comparator 600 may determine that a third element(e.g., inputParser) is object data, and may select, for the thirdelement, comparison logic 830 for object data (e.g., ObjectComparator).

Although result 820 may include a simple data structure, comparator 600may permit comparison, using appropriate comparison logic 830, of datastructures that may be significantly more complex with multiple levelsof nesting. Furthermore, comparator 600 may ensure that numeric and/ornon-numeric values are compared with appropriate tolerances, and thatthe tolerances propagate to all numeric and/or non-numeric values in adata structure. Alternatively, or additionally, comparator 600 may applymodifiers to comparisons for non-numeric data types. For example,comparator 600 may compare strings without regard to whitespace or casedifferences, may compare numeric values without regard to class (type)differences, etc. The modifiers may be propagated to appropriate valuesin a data structure.

In one example implementation, comparator list 810 may correspond to alist (e.g., the ComparatorList) of comparison logic 830 that may beselected for a comparison operation. The ComparatorList may provide thelist of comparison logic 830 to comparator component 600 (e.g., theContainerComparator) before the ContainerComparator performs thecomparison operation. The ContainerComparator may then have access tothe full list of comparison logic 830 and may therefore compare any datasupported by the ComparatorList. The ContainerComparator may support, inrecursion, any data that is support the ComparatorList.

Although FIG. 8 shows example operations capable of being performed bycomparator component 600, in other implementations, comparator component600 may perform fewer operations, different operations, and/oradditional operations than depicted in FIG. 8. Alternatively, oradditionally, one or more components of FIG. 8 may perform one or moreother tasks described as being performed by one or more other componentsof FIG. 8.

Example Process

FIGS. 9 and 10 are flow charts of an example process 900 for comparingdisparate data in arbitrarily complex data structures. In oneimplementation, process 900 may be performed by client device 210/TCE240. Alternatively, or additionally, process 900 may be performed byanother device or a group of devices separate from or including clientdevice 210/TCE 240, such as server device 220.

As shown in FIG. 9, process 900 may include receiving a first resultthat includes first data (block 910), and receiving a second result thatincludes second data (block 920). For example, in an implementationdescribed above in connection with FIG. 5, testing component 510 mayreceive results generated by TCE code 520. TCE code 520 may be executedto generate first result 530 that includes first data. TCE code 520 mayagain be executed to generate second result 540 that includes seconddata. In one example, first result 530 may be generated by a simulationversion of TCE code 520, and second result 540 may be generated by anexecutable version of TCE code 520. The first data and the second datamay be the same type or different types, and may include floating pointvalues, cell arrays, structure arrays, simulation output data, numericdata, logical data, string data, object data, etc.

As further shown in FIG. 9, process 900 may include determining whethera comparator supports the first data and the second data (block 930). Ifthe comparator does not support the first data and/or the second data(block 930—NO), process 900 may include providing an indication ofunsupported data (block 940). For example, in an implementationdescribed above in connection with FIG. 5, testing component 510 maydetermine whether a comparator, of testing component 510, supports thefirst data and the second data. If the comparator does not support thefirst data and/or the second data, testing component 510 may provide(e.g., display) an indication of unsupported data in first result 530and/or second result 540.

Returning to FIG. 9, if the comparator supports the first data and thesecond data (block 930—YES), process 900 may include utilizing thecomparator to select comparison logic for the first result and thesecond result (block 950), and comparing the first result and the secondresult, with the comparison logic, to determine if the first result isequal or unequal to the second result (block 960). For example, in animplementation described above in connection with FIG. 5, if thecomparator supports the first data and the second data, testingcomponent 510 may utilize the comparator to select comparison logic thatis appropriate for the first data of first result 530 and the seconddata of second result 540. Testing component 510 may compare firstresult 530 and second result 540, using the selected comparison logicand based on test case 550, to determine whether first result 530 isequal to second result 540.

As further shown in FIG. 9, process 900 may include outputting and/orstoring the determination of equality or inequality (block 970). Forexample, in an implementation described above in connection with FIG. 5,testing component 510 may output determination 560 of whether firstresult 530 is equal to second result 540. If first result 530 equalssecond result 540, testing component 510 may output (e.g., display)and/or store, as determination 560, diagnostic information 570indicating the equivalence of first result 530 and second result 540. Iffirst result 530 does not equal second result 540, testing component 510may output (e.g., display) and/or store, as determination 560,diagnostic information 570 indicating the nonequivalence of first result530 and second result 540.

Process block 950 may include the process blocks depicted in FIG. 10. Asshown in FIG. 10, process block 950 may include determining firstcomparison logic, of a comparator list, that matches the first data(block 1000), determining second comparison logic, of the comparatorlist, that matches the second data (block 1010), and outputting thefirst comparison logic and the second comparison logic as the selectedcomparison logic for the first result and the second result (block1020). For example, in an implementation described above in connectionwith FIG. 5, the comparator may include a list of comparison logic forcell arrays, structure arrays, numeric data, logical data, string data,object data, etc. Testing component 510 may determine first comparisonlogic of the list that matches the first data, and may determine secondcomparison logic of the list that matches the second data. Thecomparator may output the first comparison logic as the selectedcomparison logic that is appropriate for the first data of first result530. The comparator may output the second comparison logic as theselected comparison logic that is appropriate for the second data ofsecond result 540.

CONCLUSION

Systems and/or methods described herein may compare disparate data inarbitrarily complex data structures in order to determine whether thedata structures are equivalent (e.g., pass or fail a qualificationtest). In one example, the systems and/or methods may provide acomparator API that enables a tester of program code to compare complexdata structures (e.g., cell arrays, structure arrays, etc.). The datastructures may include many different types of data and each data typemay require different methods of comparison. The single comparator APImay recursively compare all elements of a data structure so that eachelement may utilize comparison logic that is appropriate for thatelement.

The foregoing description of implementations provides illustration anddescription, but is not intended to be exhaustive or to limit theimplementations to the precise form disclosed. Modifications andvariations are possible in light of the above teachings or may beacquired from practice of the implementations.

For example, while series of blocks have been described with regard toFIGS. 9 and 10, the blocks and/or the order of the blocks may bemodified in other implementations. Further, non-dependent blocks may beperformed in parallel.

It will be apparent that example aspects, as described above, may beimplemented in many different forms of software, firmware, and hardwarein the implementations illustrated in the figures. The actual softwarecode or specialized control hardware used to implement these aspectsshould not be construed as limiting. Thus, the operation and behavior ofthe aspects were described without reference to the specific softwarecode—it being understood that software and control hardware could bedesigned to implement the aspects based on the description herein.

Further, certain portions of the implementations may be implemented as a“component” that performs one or more functions. This component mayinclude hardware, such as a processor, an application-specificintegrated circuit (ASIC), or a field-programmable gate array (FPGA), ora combination of hardware and software.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of the specification. In fact, many ofthese features may be combined in ways not specifically recited in theclaims and/or disclosed in the specification. Although each dependentclaim listed below may directly depend on only one other claim, thedisclosure of the specification includes each dependent claim incombination with every other claim in the claim set.

No element, act, or instruction used in the present application shouldbe construed as critical or essential unless explicitly described assuch. Also, as used herein, the article “a” is intended to include oneor more items. Where only one item is intended, the term “one” orsimilar language is used. Further, the phrase “based on” is intended tomean “based, at least in part, on” unless explicitly stated otherwise.

What is claimed is:
 1. A device comprising: one or more processors to:receive a first result generated by code, the first result includingfirst data, receive a second result generated by the code, the secondresult including second data, determine whether a comparator supportsthe first data and the second data, utilize, if the comparator supportsthe first data and the second data, the comparator to select comparisonlogic for the first data of the first result and for the second data ofthe second result, compare, if the comparator supports the first dataand the second data, the first result and the second result, using theselected comparison logic, to obtain a determination of whether thefirst result is equivalent to the second result, output or store, if thecomparator supports the first data and the second data, thedetermination of whether the first result is equivalent to the secondresult; and provide an indication of unsupported data if the comparatordoes not support the first data or the second data.
 2. The device ofclaim 1, where, when utilizing the comparator, the one or moreprocessors are further to: determine first comparison logic of a list,of the comparator, that matches the first data, determine secondcomparison logic of the list that matches the second data, and outputthe first comparison logic and the second comparison logic as theselected comparison logic for the first data of the first result and forthe second data of the second result, respectively.
 3. The device ofclaim 1, where each of the first data and the second data include one ormore of: a floating point value, a cell array, a structure array,simulation output data, numeric data, logical data, string data, orobject data.
 4. The device of claim 1, where the selected comparisonlogic includes one or more of: comparison logic for floating or fixedpoint values, comparison logic for cell arrays, comparison logic forstructure arrays, comparison logic for simulation output data,comparison logic for numeric data, comparison logic for logical data,comparison logic for string data, or comparison logic for object data.5. The device of claim 1, where the first result includes multiple data,and where the one or more processors are further to: utilize thecomparator recursively to select multiple comparison logic for themultiple data of the first result.
 6. The device of claim 1, where thesecond result includes multiple data, and where the one or moreprocessors are further to: utilize the comparator recursively to selectmultiple comparison logic for the multiple data of the second result. 7.A method, comprising: receiving a first result generated by code, thefirst result including first data, and the receiving the first resultbeing performed by one or more devices; receiving a second resultgenerated by the code, the second result including second data, and thereceiving the second result being performed by the one or more devices;determining whether a comparator supports the first data and the seconddata, the determining being performed by the one or more devices;utilizing, if the comparator supports the first data and the seconddata, the comparator to select comparison logic for the first data ofthe first result and for the second data of the second result, theutilizing the comparator being performed by the one or more devices ifthe comparator supports the first data and the second data; comparing,if the comparator supports the first data and the second data, the firstresult and the second result, using the selected comparison logic, toobtain a determination of whether the first result is equivalent to thesecond result, the comparing the first result and the second resultbeing performed by the one or more devices if the comparator supportsthe first data and the second data; and outputting or storing, if thecomparator supports the first data and the second data, thedetermination of whether the first result is equivalent to the secondresult, the outputting or storing the determination being performed bythe one or more devices if the comparator supports the first data andthe second data; and providing an indication of unsupported data if thecomparator does not support the first data or the second data, theproviding the indication being performed by the one or more devices ifthe comparator does not support the first data and the second data. 8.The method of claim 7, where utilizing the comparator further comprises:determining first comparison logic of a list, of the comparator, thatmatches the first data; determining second comparison logic of the listthat matches the second data; and outputting the first comparison logicand the second comparison logic as the selected comparison logic for thefirst data of the first result and for the second data of the secondresult, respectively.
 9. The method of claim 7, where each of the firstdata and the second data include one or more of: a floating point value,a cell array, a structure array, simulation output data, numeric data,logical data, string data, or object data.
 10. The method of claim 7,where the selected comparison logic includes one or more of: comparisonlogic for floating or fixed point values, comparison logic for cellarrays, comparison logic for structure arrays, comparison logic forsimulation output data, comparison logic for numeric data, comparisonlogic for logical data, comparison logic for string data, or comparisonlogic for object data.
 11. The method of claim 7, where the first resultincludes multiple data, and where the method further comprises:utilizing the comparator recursively to select multiple comparison logicfor the multiple data of the first result.
 12. The method of claim 7,where the second result includes multiple data, and where the methodfurther comprises: utilizing the comparator recursively to selectmultiple comparison logic for the multiple data of the second result.13. The method of claim 7, further comprising: receiving a test case tobe applied to the first result and the second result, the test casespecifying a test to obtain the determination of whether the firstresult is equivalent to the second result, and the comparing beingperformed based on the test case.
 14. One or more non-transitorycomputer-readable media storing instructions, the instructionscomprising: one or more instructions that, when executed by a processorof a device, cause the processor to: receive a first result generated bycode, the first result including first data, receive a second resultgenerated by the code, the second result including second data,determine whether a comparator supports the first data and the seconddata, provide an indication of unsupported data when the comparator doesnot support the first data or the second data, utilize, if thecomparator supports the first data and the second data, the comparatorto select comparison logic for the first data of the first result andfor the second data of the second result, compare, if the comparatorsupports the first data and the second data, the first result and thesecond result, using the selected comparison logic, to obtain adetermination of whether the first result is equivalent to the secondresult, output or store, if the comparator supports the first data andthe second data, the determination of whether the first result isequivalent to the second result, and provide an indication ofunsupported data if the comparator does not support the first data orthe second data.
 15. The one or more non-transitory computer-readablemedia of claim 14, where the instructions further comprise: one or moreinstructions that, when executed by the processor, cause the processorto: determine first comparison logic of a list, of the comparator, thatmatches the first data, determine second comparison logic of the listthat matches the second data, and output the first comparison logic andthe second comparison logic as the selected comparison logic for thefirst data of the first result and for the second data of the secondresult, respectively.
 16. The one or more non-transitorycomputer-readable media of claim 14, where each of the first data andthe second data include one or more of: a floating point value, a cellarray, a structure array, simulation output data, numeric data, logicaldata, string data, or object data.
 17. The one or more non-transitorycomputer-readable media of claim 14, where the selected comparison logicincludes one or more of: comparison logic for floating or fixed pointvalues, comparison logic for cell arrays, comparison logic for structurearrays, comparison logic for simulation output data, comparison logicfor numeric data, comparison logic for logical data, comparison logicfor string data, or comparison logic for object data.
 18. The one ormore non-transitory computer-readable media of claim 14, where the firstresult includes multiple data, and where the instructions furthercomprise: one or more instructions that, when executed by the processor,cause the processor to: utilize the comparator recursively to selectmultiple comparison logic for the multiple data of the first result. 19.The one or more non-transitory computer-readable media of claim 14,where the second result includes multiple data, and where theinstructions further comprise: one or more instructions that, whenexecuted by the processor, cause the processor to: utilize thecomparator recursively to select multiple comparison logic for themultiple data of the second result.
 20. The one or more non-transitorycomputer-readable media of claim 14, where the first data is of a firsttype, where the second data is of a second type, and where the secondtype is different from the first type.